Repository navigation
fix: show image output price for AI Gateway image models - #6910
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
andrelandgraf
left a comment
There was a problem hiding this comment.
Aileen review: approve (fix, medium risk, ae536ee)
Verdict
Approve. The PR maps the gateway's pricing.image_output to cost.output_image (USD per 1M tokens) and shows it in the model table, the llms markdown table, and the per-model detail page. The evidence holds against the live gateway.
Verification
- Live gateway: on a throwaway project in
org-old-flower-82714815(deleted afterwards),GET /v1/modelslisted 68 models. Onlygemini-3-pro-image(image_output: "0.00012") andgemini-3-1-flash-image("0.00006") carry an image rate, and nooverridesentry has one. Running the PR'sbuildCatalog+validateCatalogon that listing returns no errors and produces{"input":2,"output":12,"output_image":120}and{"input":0.5,"output":3,"output_image":60}. Compared with the liveneon.com/models.json, those twocostblocks are the only difference. - models.dev's
Costschema is.strict()with no image key, so excludingoutput_imagefromclassifyDriftis justified. Against the live models.devneonprovider, both image models are currently inawaitingUpstream, so today the exclusion only keeps that list from gaining a false difference later. npm run test:unit:run -- --retry=2 --maxWorkers=4 --testTimeout=30000 --hookTimeout=60000: 115 files and 1543 tests pass, which matches the description. ESLint andprettier --checkpass on the 12 changed files.- The PR's screenshots show the second
image $Nline only on the two image models, with text-only rows unchanged, and the detail page shows theImage output /Msection. - A second-opinion reviewer (Claude Opus 5.5) reported no blocking findings.
Scope
The diff covers one concern: one mapping line, one allowed rate key, one drift exclusion, and three render branches, each with a focused test. data.json is unchanged, and the build refresh adds the key. The task's "Latest" view does not exist in the code or on live neon.com, and the description says so. It also leaves route documentation unchanged because capabilities.json has no measurements for these models. Both are reasonable. No decision in force applies.
Residual risk (why medium)
cost.output_imageis the firstcostkey on/models.jsonthat models.dev's schema doesn't define, androute.jssays the endpoint can be read by the same code that reads models.dev. A downstream reader that parsescostwith models.dev's strict schema (the console model picker is one consumer, and its code isn't available here) would reject these two entries. Readers that ignore unknown keys, and the@neon/ai-sdk-providerdrift test, are unaffected.- Image rates on context-tier overrides would be dropped, because the tiers are built with
rates(). No model has one today. - Follow-ups outside this PR, also listed in its description: add a note to the
add-model-to-ai-gatewayskill to leaveoutput_imageout of the models.dev[cost], and have Orbitcatalog-mirrorignore the key.
Nits (non-blocking)
model-rows.jschecks=== undefined, whiletoCostuses!= null. An explicitnullwould renderimage —, butvalidateCatalogrejectsnull, so this can't happen today.- On the landing desktop table,
image $60wraps to two lines in the narrow Output /M column.
|
Waiting for Andre: category fix needs Andre. Merge it when it is ready, or comment to have Aileen change it. |
This PR comes from a request from Andre.
Problem
gemini-3-pro-imageandgemini-3-1-flash-imagebill image output tokens at a separate rate. The gateway'sGET /v1/modelscarries that rate, but neon.com publishes only the text output rate:The build-time refresh (
scripts/lib/ai-gateway-models.mjs) maps onlyprompt,completion, and the cache rates, soimage_outputis dropped. The model table on/ai-gateway#modelsand/docs/ai-gateway/overview, the per-model docs pages, and the llms markdown mirror then show $12/M for Gemini 3 Pro Image, while image output costs $120/M. Gemini 3.1 Flash Image shows $3/M; its image output costs $60/M.No other listed model has
image_output. GPT-6.1 Terra and Gemini 4 Carbon listoutput_modalities: ["text"].Solution
The fix belongs in the catalog pipeline and the shared row builder, so every surface that reads
/models.jsongets it:toCost()mapspricing.image_outputto a newcost.output_imagekey, in USD per 1M tokens.validateCatalogaccepts the key. Without that, the refresh would reject the catalog and keep the committed file.Costschema is strict and has no image rate, sooutput_imageexists only on neon.com.classifyDriftleaves it out of the models.dev mirror comparison, so the scheduled mirror check doesn't report a difference that models.dev can never close.toRow()addscostOutputImageLabel, which isnullfor models without an image rate. All three renderers read it:$12$12, thenimage $120on a second, smaller line$12$12 (image $120)/docs/ai-gateway/models/<id>Output /MsectionImage output /Msection afterOutput /MThe second line wraps inside the existing Output /M column. Column widths are unchanged, so text-only rows lay out the same as before. Sorting by Output /M still uses the text rate.
The committed
data.jsonis unchanged. The build refresh writes the new key on the next production or preview build.Verification
npm run test:unit:run -- --retry=2 --maxWorkers=4 --testTimeout=30000 --hookTimeout=60000: 115 files and 1543 tests pass. New tests cover the mapper, the validator (it rejects a string or negativeoutput_image), the drift exclusion, row labels, the table cell, the markdown cell, and the detail page sections. The existing text-only assertions are unchanged and still pass.node scripts/verify-agent-endpoints.mjspasses 13/13 endpoints and 68/68 checks.npm run check:models-syncexits 0.scripts/update-models-from-ai-gateway.mjswrote{"input":2,"output":12,"output_image":120}forgemini-3-pro-imageand{"input":0.5,"output":3,"output_image":60}forgemini-3-1-flash-image. No other model changed, compared with a refresh on unchanged code.next devserved/models.jsonwith the new key, and the regenerated.mdpage forgemini-3-pro-imagehas## Image output /Mwith$120.?q=gemini-3, for the landing table (dark, desktop and mobile), the docs table (light and dark, desktop and mobile), and the Gemini 3 Pro Image detail page (light and dark). After the change, both image models showimage $60andimage $120under the text price, and the detail page has an Image output /M section. The text-only rows above the first image model are pixel-identical before and after in the landing and docs desktop captures. The browser console showed no errors.Left open
release_dateis 2025-11-20. The models do not appear under the Image tab. That tab filters on measuredimageGenerationinsrc/app/models/capabilities.json, which has no records forgemini-3-pro-image,gemini-3-1-flash-image, orgpt-6-1-terra. Those records come from the conformance probe inneondatabase/neon-console-code-examples. Until they exist, these models also have no verified endpoints or code examples on their detail pages, and/modelsomits them.capabilities.jsonhas no route data for the two image models, so the page states that no verified endpoints are available. The gateway listing'sdialectsfield for both is["chat_completions", "gemini_generate_content"]. The route docs stay unchanged until the probe measures image output on each route. Gemini 4 Carbon's measured routes (chat completions with array content, plus the native Gemini route) are already in the docs.neonprovider, leaveoutput_imageout of[cost], because models.dev's schema rejects it. Theadd-model-to-ai-gatewayskill says to copy[cost]from the website entry; it needs a note about this exception. Orbit'scatalog-mirrorjob comparescostseparately from this repo and will report the extra key as a difference unless it also ignores it.Review media
Opened by Aileen job
aileen/neon-dx/task/task-2026-11-20t20-35-18-844z/cli-run-2026-11-20t20-35-18-844z(factoryneon-dx).Next: Aileen's reviewer classifies and reviews this PR; the factory policy decides whether it merges without Andre.
Task: